Processing Time Analysis¶

This notebook allows you to select and visualize processing time and read time data from marker location files.

In [15]:
import pandas as pd
import plotly.graph_objects as go
from plotly.subplots import make_subplots
import plotly.io as pio
import os
from pathlib import Path

# Configure Plotly for offline use in exports
pio.renderers.default = "notebook"

Setup¶

Configure the base directory and define functions to find folders containing MarkerLocationsGA_CouchShift files.

In [16]:
# Base directory for data
base_dir = r"C:\Users\kankean.kandasamy\Repo\KIM-QA-Analysis\Lyrebird Data"

# Column names for MarkerLocationsGA files (even if headers are missing)
GA_COLUMNS = [
    'Frame No', 'Time (sec)', 'Gantry', 
    'Marker_0_AP', 'Marker_0_LR', 'Marker_0_SI',
    'Marker_1_AP', 'Marker_1_LR', 'Marker_1_SI',
    'Marker_2_AP', 'Marker_2_LR', 'Marker_2_SI',
    'Marker_0_X', 'Marker_0_Y', 'Marker_1_X', 'Marker_1_Y', 'Marker_2_X', 'Marker_2_Y',
    'Marker_0_Correlation', 'Marker_0_Segmentation',
    'Marker_1_Correlation', 'Marker_1_Segmentation',
    'Marker_2_Correlation', 'Marker_2_Segmentation',
    'Joint Template', 'Processing Time (ms)', 'Read Time (ms)',
    'HeaderTimeTag', 'Filename'
]

# Find all subdirectories
def find_folders(base_path):
    folders = []
    for root, dirs, filenames in os.walk(base_path):
        # Check if this directory contains MarkerLocationsGA_CouchShift files
        has_ga_files = any('MarkerLocationsGA_CouchShift' in f and f.endswith('.txt') for f in filenames)
        if has_ga_files:
            relative_path = os.path.relpath(root, base_path)
            folders.append((relative_path, root))
    return sorted(folders)

# Find all MarkerLocationsGA_CouchShift files in a folder
def find_ga_files_in_folder(folder_path):
    files = []
    for filename in os.listdir(folder_path):
        if 'MarkerLocationsGA_CouchShift' in filename and filename.endswith('.txt'):
            full_path = os.path.join(folder_path, filename)
            files.append((filename, full_path))
    return sorted(files)

available_folders = find_folders(base_dir)
print(f"Found {len(available_folders)} folders with MarkerLocationsGA_CouchShift files")
Found 17 folders with MarkerLocationsGA_CouchShift files

Plotting Functions¶

Define the data reading and plotting logic.

In [17]:
def plot_file(file_path, x_axis_col='Time (sec)'):
    """Read a single data file and create interactive plots"""
    try:
        # Try reading with headers first
        try:
            df = pd.read_csv(file_path, skipinitialspace=True)
            df.columns = df.columns.str.strip()
            # Check if we have the expected columns
            if 'Processing Time (ms)' not in df.columns:
                raise ValueError("Missing expected columns")
        except:
            # Read without headers and assign column names
            df = pd.read_csv(file_path, header=None, names=GA_COLUMNS, skipinitialspace=True)
        
        # Extract relevant columns
        x_data = df[x_axis_col]
        processing_time = df['Processing Time (ms)']
        read_time = df['Read Time (ms)']
        
        # Create subplots with 2 rows
        fig = make_subplots(
            rows=2, cols=1,
            subplot_titles=('Processing Time', 'Read Time'),
            vertical_spacing=0.12,
            shared_xaxes=True
        )
        
        # Add Processing Time trace (top panel)
        fig.add_trace(
            go.Scatter(
                x=x_data,
                y=processing_time,
                mode='lines+markers',
                name='Processing Time',
                line=dict(color='#1f77b4', width=2),
                marker=dict(size=4),
                hovertemplate=f'{x_axis_col}: %{{x}}<br>Processing Time: %{{y:.2f}} ms<extra></extra>'
            ),
            row=1, col=1
        )
        
        # Add Read Time trace (bottom panel)
        fig.add_trace(
            go.Scatter(
                x=x_data,
                y=read_time,
                mode='lines+markers',
                name='Read Time',
                line=dict(color='#ff7f0e', width=2),
                marker=dict(size=4),
                hovertemplate=f'{x_axis_col}: %{{x}}<br>Read Time: %{{y:.2f}} ms<extra></extra>'
            ),
            row=2, col=1
        )
        
        # Update axes labels
        fig.update_xaxes(title_text=x_axis_col, row=2, col=1)
        fig.update_yaxes(title_text='Processing Time (ms)', row=1, col=1)
        fig.update_yaxes(title_text='Read Time (ms)', row=2, col=1)
        
        # Update layout with filename
        filename = os.path.basename(file_path)
        folder_name = os.path.basename(os.path.dirname(file_path))
        fig.update_layout(
            height=800,
            showlegend=True,
            hovermode='x unified',
            title=dict(
                text=f'<b>{folder_name} - {filename}</b>',
                x=0.5,
                xanchor='center'
            ),
            font=dict(size=12)
        )
        
        # Enable pan and zoom
        fig.update_xaxes(rangeslider_visible=False)
        
        return fig, df
        
    except Exception as e:
        print(f"Error reading file {os.path.basename(file_path)}: {e}")
        return None, None


def plot_folder(folder_path, x_axis='Time (sec)'):
    """
    Plot all MarkerLocationsGA_CouchShift files in a folder.
    
    Parameters:
    -----------
    folder_path : str
        Relative path to folder (e.g., "2025-10-28/Case4 - iso2 repeat")
        or absolute path
    x_axis : str
        'Time (sec)' or 'Frame No'
    """
    # Handle both relative and absolute paths
    if not os.path.isabs(folder_path):
        full_path = os.path.join(base_dir, folder_path)
    else:
        full_path = folder_path
    
    if not os.path.exists(full_path):
        print(f"Error: Folder not found: {full_path}")
        return
    
    # Find all GA files in folder
    ga_files = find_ga_files_in_folder(full_path)
    
    if not ga_files:
        print(f"No MarkerLocationsGA_CouchShift files found in: {folder_path}")
        return
    
    folder_name = os.path.basename(full_path)
    print(f"Processing {len(ga_files)} file(s) from: {folder_name}\n")
    
    for filename, file_path in ga_files:
        print(f"Loading: {filename}")
        fig, df = plot_file(file_path, x_axis)
        
        if fig is not None:
            fig.show()
            print(f"\nData Summary for {filename}:")
            print(f"Total Frames: {len(df)}")
            print(f"\nProcessing Time (ms):")
            print(f"  Mean: {df['Processing Time (ms)'].mean():.2f}")
            print(f"  Median: {df['Processing Time (ms)'].median():.2f}")
            print(f"  Min: {df['Processing Time (ms)'].min():.2f}")
            print(f"  Max: {df['Processing Time (ms)'].max():.2f}")
            print(f"\nRead Time (ms):")
            print(f"  Mean: {df['Read Time (ms)'].mean():.2f}")
            print(f"  Median: {df['Read Time (ms)'].median():.2f}")
            print(f"  Min: {df['Read Time (ms)'].min():.2f}")
            print(f"  Max: {df['Read Time (ms)'].max():.2f}")
            print("\n" + "="*80 + "\n")


def list_folders():
    """List all available folders containing MarkerLocationsGA_CouchShift files"""
    print(f"Available folders ({len(available_folders)}):\n")
    for i, (name, path) in enumerate(available_folders, 1):
        print(f"{i:2d}. {name}")
    print("\nUsage: plot_folder('folder_path', x_axis='Time (sec)')")

Usage¶

Available Functions¶

  1. list_folders() - List all available folders with data files
  2. plot_folder(folder_path, x_axis='Time (sec)') - Plot all files in a folder
    • folder_path: Relative path from base directory (e.g., "2025-10-28/Case4 - iso2 repeat")
    • x_axis: Either 'Time (sec)' or 'Frame No'
  3. plot_file(file_path, x_axis_col='Time (sec)') - Plot a single file

Interactive Features¶

All plots use Plotly with interactive controls:

  • Pan: Click and drag
  • Zoom: Scroll wheel or box select
  • Reset: Double-click the plot
  • Hover: Move mouse over data points for details

Step 1: List Available Folders¶

In [18]:
list_folders()
Available folders (17):

 1. 2025-03-05 CTRO Static Tests Repeat
 2. 2025-03-09 CTRO Static Tests with ImageX Phantom\Expt_9_03_2035\Couch_shift_5_5_6
 3. 2025-03-09 CTRO Static Tests with ImageX Phantom\Expt_9_03_2035\Iso_volumeview_motionview
 4. 2025-03-09 CTRO Static Tests with ImageX Phantom\Expt_9_03_2035\Marker_migrstion
 5. 2025-03-18 CTRO Robot Repeat - Static Erratic and Persistent\Erratic
 6. 2025-10-28\Case1 - iso1
 7. 2025-10-28\Case2 - shift left 5 mm
 8. 2025-10-28\Case3 - 5mm offset start, interruprt and shift according to KIM
 9. 2025-10-28\Case4 - iso2 repeat
10. 2025-11-17
11. 2025-11-17\Lat shift present
12. 2025-11-20
13. 2025-11-26
14. 2025-12-02
15. 2025-12-02\Nice
16. Static Test
17. Trt_int_prersistant_excursion

Usage: plot_folder('folder_path', x_axis='Time (sec)')

Step 2: Plot Data from a Folder¶

Uncomment and modify the example below to plot your data:

In [19]:
# Example: Plot with Time on x-axis (default)
plot_folder("2025-11-20")
plot_folder("2025-03-05 CTRO Static Tests Repeat")
# Example: Plot with Frame No on x-axis
# plot_folder("2025-10-28\\Case4 - iso2 repeat", x_axis='Frame No')
Processing 2 file(s) from: 2025-11-20

Loading: MarkerLocationsGA_CouchShift_0.txt
Data Summary for MarkerLocationsGA_CouchShift_0.txt:
Total Frames: 128

Processing Time (ms):
  Mean: 279.20
  Median: 203.00
  Min: 0.00
  Max: 13648.00

Read Time (ms):
  Mean: 200.49
  Median: 168.50
  Min: 100.00
  Max: 785.00

================================================================================

Loading: MarkerLocationsGA_CouchShift_1.txt
Data Summary for MarkerLocationsGA_CouchShift_1.txt:
Total Frames: 463

Processing Time (ms):
  Mean: 232.37
  Median: 227.00
  Min: 144.00
  Max: 442.00

Read Time (ms):
  Mean: 251.21
  Median: 198.00
  Min: 98.00
  Max: 1118.00

================================================================================

Processing 1 file(s) from: 2025-03-05 CTRO Static Tests Repeat

Loading: MarkerLocationsGA_CouchShift_0.txt
Data Summary for MarkerLocationsGA_CouchShift_0.txt:
Total Frames: 844

Processing Time (ms):
  Mean: 234.47
  Median: 225.50
  Min: 0.00
  Max: 6297.00

Read Time (ms):
  Mean: 130.63
  Median: 101.00
  Min: 41.00
  Max: 515.00

================================================================================